-
Notifications
You must be signed in to change notification settings - Fork 529
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Paper: AI driven WatermarkingTechnique for Safegaurding Text Integrity #928
base: 2024
Are you sure you want to change the base?
Paper: AI driven WatermarkingTechnique for Safegaurding Text Integrity #928
Conversation
c693614
to
c9e172b
Compare
c9e172b
to
43cd8f6
Compare
Hi @hongsupshin, I am not sure why the ID is returning null. Can you help please? |
Hey @Atharva-Rasane - thanks for the submission! The check is expecting your content at |
@Atharva-Rasane can you make the update requested here |
Curvenote Preview
|
Apologies for the delay, I have made the changes. Do you need me to squash the changes into 1 commit or is this fine now? |
this should be okay. thanks |
Hello @Atharva-Rasane I'm Tolulade, Data & ML Architect, founder @ Reispar Technologies. I'm happy to be one of the reviewers for your paper. |
@Atharva-Rasane @Tolulade-A please review and update the paper title with correct spelling and grammar. 🙂 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hi @Atharva-Rasane I have added some comments for you to review.
papers/atharva_rasane/main.md
Outdated
|
||
In this paper, we explore text watermarking as a potential solution. We examine various methods, focusing on plain ASCII text in English. Our goal is to investigate different techniques, including physical watermarking (e.g., UniSpaCh by Por et al.), where text is modified to hide a binary message using Unicode Spaces, and logical watermarking (e.g., word context proposed by Jalil et al.), where a watermark key is generated via a defined process. While logical watermarking is difficult to break, it is not detectable without prior knowledge of the algorithm and parameters used. Conversely, physical watermarks are easily detected but also easy to break. | ||
|
||
This paper presents a unique physical watermarking technique based on word substitution to address these challenges. The core idea is that AI models consistently produce the same output for the same input. Initially, we replaced every ith word with a "[MASK]," then used a BERT model to predict the most probable token in place of "[MASK]." The resulting text constitutes the watermarked text. To verify, we reran the algorithm on the watermarked text and compared the input and output for similarity. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Atharva-Rasane you can add more context and clarity where the ith
word is used, is this a random word sampling?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello, yes it is a random index value. We could choose i-th to be say the 5-th word. If I added a hyphen, would that help a bit more with the clarity?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now I get you, adding a hypen would help and a bit of additional context like you did with this reply.
papers/atharva_rasane/main.md
Outdated
|
||
This paper presents a unique physical watermarking technique based on word substitution to address these challenges. The core idea is that AI models consistently produce the same output for the same input. Initially, we replaced every ith word with a "[MASK]," then used a BERT model to predict the most probable token in place of "[MASK]." The resulting text constitutes the watermarked text. To verify, we reran the algorithm on the watermarked text and compared the input and output for similarity. | ||
|
||
The Python implementation of the algorithm in this paper employes models from the HuggingFace Transformer Library, namely "bert-base-uncased" and "distilroberta-base". The "[MASK]" placeholder was generated by splitting the input string using the `split()` function and then replacing every ith element in the list with "[MASK]". This modified list served as the input text for the BERT model, where the output corresponding to each "[MASK]" was replaced accordingly. Finally, applying the join() function to the list produces the watermarked text. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Atharva-Rasane the ith word is also used here, kindly add more context, is this a random word sampling ?
papers/atharva_rasane/main.md
Outdated
|
||
With the majority of the internet and tools like ChatGPT and Bard being text-focused, we need to realize the importance of identifying the source of text whether due to copyright or to differentiate between AI-generated text and Human written text to prevent the flow of misinformation. The standard of detecting AI-generated text is with the use of another ML classifier which needs to be constantly trained on the latest AI-generated text data. This approach has a few drawbacks, one of which is the ever-changing nature of AI-generated text where we have bigger and better models that are giving more human-like text being released faster then ever before and thus we need a more standard/concrete approach, one that can be used regardless of the AI model i.e. we need a method of identifying that doesn't depend on one generating the text. One such approach is via the use of a watermark. | ||
|
||
Watermarks are an identifying pattern used to identify the origin of the data. In this case, we specifically want to focus on text watermarking (watermarking of plain text). Text watermarking can broadly be classified into 2 types Logical Embedding and Physical Embedding which in turn can be classified further. Logical Embedding involves the user generating a watermark key by some logic from the input text. Note that this means that the input text is not altered and the user instead keeps the generated watermark key to identify the text. Physical Embedding involves the user altering the input text itself to insert a message into it and the user instead runs an algorithm to find this message to identify the text. In this paper, we will propose an algorithm to watermark text using BERT (Bidirectional Encoder Representations from Transformers), a model introduced by Google whose main purpose is to replace a special symbol [MASK] with the most probable word given the context. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Atharva-Rasane is MASK here an acronymn? If it is, it can be spelt in full.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MASK here is not an acronym but signifies a special token which BERT model then identifies and replaces - This token and its usage was pulled directly from one of the cited papers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did I miss it ? Was there any place you elaborated slightly on what Mask is ? This could help simplify things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have incuded it in the inroduction:
"In this paper, we will propose an algorithm to watermark text using BERT (Bidirectional Encoder Representations from Transformers), a model introduced by Google whose main purpose is to replace a special symbol "[MASK]" with the most probable word given the context."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @Atharva-Rasane I have taken note of the changes, but I will still go through them again and get back to you if there's any other feedback
886ef5c
to
33e4b62
Compare
33e4b62
to
37b266e
Compare
papers/atharva_rasane/main.md
Outdated
# Ensure that this title is the same as the one in `myst.yml` | ||
title: AI-Driven Watermarking Technique for Safeguarding Text Integrity in the Digital Age | ||
abstract: | | ||
The internet's growth has led to a surge in text usage. Now, with public access to generative AI models like ChatGPT/Bard, identifying the source is vital. This is crucial due to concerns about copyright infringement and plagiarism. Moreover, it's essential to differentiate AI-generated text to curb misinformation from AI model hallucinations. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please replace "it's" with "it is". "It is" is formal.
papers/atharva_rasane/main.md
Outdated
--- | ||
|
||
## Introduction | ||
The growth of the internet is driven by the spread of web pages, which are written in HTML (Hyper Text Markup Language). These web pages contain large amounts of text. Almost every webpage in some form or another contains text making it a popular mode of communication whether it be blogs, posts, articles, comments etc. Text can be represented as a collection of ASCII or Unicode values, where each value corresponds to a specific character. Given the text-focused nature of the internet and tools like ChatGPT and Bard, it is crucial to identify the source of text. This helps to manage copyright issues and distinguish between AI-generated and human-written text, thereby preventing the spread of misinformation. Currently, detecting AI-generated text relies on machine learning classifiers that need frequent retraining with the latest AI-generated data. However, this method has drawbacks, such as the rapid evolution of AI models producing increasingly human-like text. Therefore, a more stable approach is needed, one that does not depend on the specific AI model generating the text. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ChatGPT or Bard, (and not "and") unless other tools are specifically ignored here. In that case, remove "like".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have Replaced and with or
papers/atharva_rasane/main.md
Outdated
## Introduction | ||
The growth of the internet is driven by the spread of web pages, which are written in HTML (Hyper Text Markup Language). These web pages contain large amounts of text. Almost every webpage in some form or another contains text making it a popular mode of communication whether it be blogs, posts, articles, comments etc. Text can be represented as a collection of ASCII or Unicode values, where each value corresponds to a specific character. Given the text-focused nature of the internet and tools like ChatGPT and Bard, it is crucial to identify the source of text. This helps to manage copyright issues and distinguish between AI-generated and human-written text, thereby preventing the spread of misinformation. Currently, detecting AI-generated text relies on machine learning classifiers that need frequent retraining with the latest AI-generated data. However, this method has drawbacks, such as the rapid evolution of AI models producing increasingly human-like text. Therefore, a more stable approach is needed, one that does not depend on the specific AI model generating the text. | ||
|
||
Watermarks are an identifying pattern used to trace the origin of the data. In this case, we specifically want to focus on text watermarking (watermarking of plain text). Text watermarking can broadly be classified into 2 types Logical Embedding and Physical Embedding which in turn can be classified further [@Atr01]. Logical Embedding involves the user generating a watermark key by some logic from the input text. Note that this means that the input text is not altered and the user instead keeps the generated watermark key to identify the text. Physical Embedding involves the user altering the input text itself to insert a message into it and the user instead runs an algorithm to find this message to identify the text. In this paper, we will propose an algorithm to watermark text using BERT (Bidirectional Encoder Representations from Transformers), a model introduced by Google whose main purpose is to replace a special symbol "[MASK]" with the most probable word given the context. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
... into 2 types, Logical Embedding and Physical Embedding, ...
Please add commas for readability.
papers/atharva_rasane/main.md
Outdated
## Proposed Model | ||
"BERT-based watermarking is based on the 5-gram approach by Lancaster[@Atr02]. However, our focus is on watermarking any text, regardless of its origin. This paper will use **bert-base-uncased** model, which finds the most probable uncased English word to replace the [MASK] token. | ||
|
||
Note that a different variant of BERT can be trained on different language datasets and thus will generate a different result and as such the unique identity to consider here is the BERT model i.e. if the user wants a unique watermark they need to train/develop the BERT model on their own. This paper is not concerned with the type of BERT model and is focused on its conceptual application for watermarking. Thus for us, BERT is a black box model that returns the most probable word given the context with the only condition being that it has a constant temperature i.e. it doesn't hallucinate (produce different results for the same input). For our purposes, you can think of the proposed algorithm as a many to one function which is responsible for converting the input text into a subset of watermarked set. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
doesn't -> does not
papers/atharva_rasane/main.md
Outdated
templ = tempd[0] | ||
temps = templ['token_str'] | ||
watermarked_words[i+4] = temps.split()[0] | ||
# print("Done ", i + 1, "th word") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider removing commented code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have removed the commented code
papers/atharva_rasane/main.md
Outdated
# Example usage | ||
text = "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems." | ||
# match_ratios = watermark_text_and_calculate_matches(text, max_offset=5) | ||
# check_significant_difference(match_ratios) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider removing commented code.
papers/atharva_rasane/main.md
Outdated
|
||
# Calculate match ratios | ||
match_ratios = watermark_text_and_calculate_matches(text, max_offset=5) | ||
result = {0: 0.5384615384615384, 1: 0.6153846153846154, 2: 0.5833333333333334, 3: 0.6666666666666666, 4: 0.5833333333333334} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider rounding the decimals to fewer digits.
papers/atharva_rasane/main.md
Outdated
(result) modified_text = "Quantum computing is example a rapidly evolving field that leverages the principles of quantum mechanics to perform random computations that are impossible for classical computers. Unlike quantum computers, which use bits as the random insert fundamental unit of , quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously according random to the principles of symmetry and entanglement, providing a significant advantage in solving complex mathematical problems." | ||
|
||
match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5) | ||
(result) match_ratios = {0: 0.5714285714285714, 1: 0.5714285714285714, 2: 0.5384615384615384, 3: 0.38461538461538464, 4: 0.7692307692307693} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider rounding the decimals to fewer digits.
papers/atharva_rasane/main.md
Outdated
Highest Match Ratio: 0.7692307692307693 | ||
Average of Other Ratios: 0.5164835164835164 | ||
T-Statistic: -5.66220858504931 | ||
P-Value: 0.010908789440745323 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider rounding the decimals to fewer digits.
Highest Match Ratio: 0.7692307692307693 | ||
Average of Other Ratios: 0.5164835164835164 | ||
T-Statistic: -5.66220858504931 | ||
P-Value: 0.010908789440745323 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above - Consider rounding the decimals to fewer digits.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have round all down to 2 decimal places
papers/atharva_rasane/main.md
Outdated
4. Potential loss of context: When words are replaced, the intended context of delivery could be altered. However, AI models are continually improving, and we hope that a well-trained model can significantly mitigate this risk. | ||
|
||
**Real-world applicability:** | ||
1. Versatility in applications: This method can be applied across various fields such as copyright protection, and content authentication, and in legal and academic settings where proof of authorship is crucial. It's particularly beneficial for managing copyrights in digital media, academic papers, and any online content where text is dynamically generated or reused. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"it's" -> it is
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I replaced all contractions in the paper
…esn't and more with their full forms
@Atharva-Rasane thanks for the submission and the subsequent edits. Tomorrow, 2nd Sept is the last day for the authors to make edits. @Tolulade-A thank you for the help with the review. Will you please take a look and get back with your final decision by Sept 9? |
sure, will do that @sanhitamj |
If you are creating this PR in order to submit a draft of your paper, please name your PR with
Paper: AI driven WatermarkingTechnique for Safegaurding Text Integrity
. An editor will then add apaper
label and GitHub Actions will be run to check and build your paper.See the project readme for more information.
Editor: Sanhita Joshi @sanhitamj
Reviewers: